Subset-selection and Ensemble Methods for Wavelet De-noising

Many nonparametric regression procedures are based on “subset selection”: they choose a subset of carriers from a large or even infinite set, and then determine the coefficients of the chosen carriers by least squares. Procedures which can be cast in this framework include Projection Pursuit, Turbo, Mars, and Matching Pursuit. Recently, considerable attention has been given to “ensemble estimators” which combine least squares estimates obtained from multiple subsets of carriers. In the parametric regression setting, such ensemble estimators have been shown to improve on the accuracy of subset selection procedures in some situations. In this paper we compare subset selection estimators and ensemble estimators in the context of wavelet de-noising. We present simulation results demonstrating that a certain class of ensemble wavelet estimators, based on the concept of “cycle spinning”, are significantly more accurate than subset selection methods. These advantages hold even when the subset selection procedures can rely on an oracle to select the optimal number of carriers. We compute ideal thresholds for translation invariant wavelet shrinkage and investigate other approaches to ensemble wavelet estimation.